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METHOD FOR ADJUSTING AN IMAGE SENSOR 



Background Information 

The present invention relates to a method and a processing unit for adjusting the 
characteristic curve of the exposure sensitivity of an image sensor, the image sensor being 
located in particular in a motor vehicle. 

5 Outdoor scenes often have a broad range of different radiation intensities. In particular in 
traffic scenes in the motor vehicle field, frequently a great difference in radiation intensities 
occurs. This becomes clear for example when a motor vehicle approaches a tunnel entrance in 
daylight. While the surroundings of the motor vehicle outside the tunnel are bright in the 
sunlight, often only weak lighting is found in the tunnel itself. With the plan to equip motor 

10 vehicles with image sensors which register the surroundings of the motor vehicle, it is 

desirable for the image sensors to process this wide range of different radiation intensities. In 
the conventional regulation of an image sensor, the image sensor is adapted to the lighting 
conditions through appropriate choice of the parameters of gain, offset, integration time, 
and/or aperture. In newer developments of image sensors, the exposure sensitivity does not 

1 5 follow a rigidly set linear or logarithmic characteristic curve, but rather the characteristic 
curve may be adjusted individually in particular in single linear segments. For example, a 
CMOS image sensor having an adjustable characteristic curve with linear segments is known 
from US 6,348,681 of February 19, 2002. Reference to a method and a processing unit for 
attaining high information content of generated images is lacking in US 6,348,681. 

20 Advantages of the Invention 

The method described below for adjusting the characteristic curve of the exposure sensitivity 
of at least one pixel of at least one image sensor has the advantage that the generated image 
information has high information content and a broad dynamic range. The method described 
below is particularly advantageous in outdoor use of the at least one image sensor, since 
25 outdoors a plurality of different radiation intensities may occur simultaneously in a scene. In 



particular, the method described below is suitable for use in a motor vehicle, since with this 
method, for example, sections of streets which are in full sunlight and simultaneously have 
areas of shadow may be mapped with high information content of the image information. In 
an advantageous manner, the method described below utilizes the additional degrees of 
5 freedom of a characteristic curve of the exposure sensitivity having segments of functions, in 
particular continuous and/or linear functions, so that the at least one image sensor captures a 
maximum of information in its images about the scene being mapped at that time. 

It is advantageous to approximate the characteristic curve of the exposure sensitivity to the 
optimal characteristic curve of the exposure sensitivity through at least one numerical 
10 approximation method and/or at least one segmenting method. In particular, approximation 
through at least one segmenting method has the advantage that the at least one segmenting 
method contributes to a simple implementation of the method on a microprocessor. 

Particularly advantageous is adjustment of the characteristic curve of the exposure sensitivity, 
the characteristic curve being formed in segments of linear functions. Characteristic curves of 
15 the exposure sensitivity having segments of linear functions contribute to the characteristic 
curves being easily realizable in the image sensor, and also enable a simple algorithmic 
implementation of the method described below in the processing unit. 

It is advantageous to adjust the characteristic curve of the exposure sensitivity of at least one 
pixel of at least one image sensor as a function of image signals from at least two image 
20 sensors, in particular from at least one stereo camera which is made up of at least two 

sensors. This procedure has the advantage that by averaging the image signals the precision 
and robustness of the method for adjusting the exposure sensitivity of at least one pixel of at 
least one image sensor is increased. 

The described advantages apply correspondingly to the method described below and/or the 
25 processing unit described below and/or the computer program described below. 

A computer program having program code means for carrying out all the steps, or at least the 
essential steps of the described method, when the program is executed on a computer, is 
advantageous. The use of a computer program facilitates quick and cost-effective adaptation 
of the method, for example to different image sensors. 
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Additional advantages derive from the following description of exemplary embodiments with 
reference to the figures, and from the dependent claims. 

Drawing 

The present invention is explained in greater detail below on the basis of the embodiments 
5 illustrated in the drawing. 

Figure 1 shows a diagram with characteristic curves of the exposure sensitivity of an 
image sensor, 

Figure 2 shows a diagram with characteristic curves of the exposure sensitivity of an 
image sensor as a function of the gain and the offset, 
10 Figure 3 shows a block diagram of the preferred exemplary embodiment. 

Detailed Description of Exemplary Embodiments 

A method and a processing unit for adjusting the characteristic curve of the exposure 
sensitivity of at least one pixel of at least one image sensor in a motor vehicle are described 

15 below. In the preferred exemplary embodiment, the optimal characteristic curve of the 

exposure sensitivity is determined from the histogram of the gray values of at least one image 
as a function of image signals from at least one image sensor. The freely definable 
characteristic curve of the image sensor having linear segments is chosen so that it agrees at 
least approximately with the optimal characteristic curve. Along with the size and/or the 

20 position of the characteristic curve of the exposure sensitivity, alternatively or in addition the 
shape of the characteristic curve is adjusted. 

Figure 1 shows a diagram with three characteristic curves 14, 16, 18 of the exposure 
sensitivity of a pixel of an image sensor. Exposure 10 is plotted on the abscissa. Exposure 10 
is plotted here in freely defined, virtual units of 0 to 100, exposure 10 being for example a 

25 measure of the radiation intensity or the illumination intensity. Here the radiation intensity is 
a physical value which indicates the radiation power per unit area, while the illumination 
intensity is the corresponding photometric value using the unit lux, which takes into account 
the spectral sensitivity of the human eye. Output signal 12 of the pixel of the image sensor is 
plotted on the ordinate as a freely defined, virtual unit from 0 to 1000. Output signal 12 may 

30 be present as a digital signal or an analog signal. As a first form of the characteristic curve of 
the exposure sensitivity, a linear characteristic curve 14 of the exposure sensitivity is plotted 
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in Figure 1. The diagram in Figure 1 also shows a logarithmic characteristic curve 16 of the 
exposure sensitivity and a characteristic curve 1 8 of the exposure sensitivity in linear 
segments. 

Figure 2 shows a diagram with four characteristic curves 20, 22, 24, 26 of the exposure 
5 sensitivity in linear segments of a pixel of an image sensor as a function of the gain and/or the 
offset. Corresponding to Figure 1, exposure 10 is plotted on the abscissa in freely defined, 
virtual units from 0 to 100. Output signal 12 of the pixel of the image sensor is plotted on the 
ordinate as a freely defined, virtual unit from 0 to 2000. In first characteristic curve 20 of the 
exposure sensitivity, the gain is 1 and the offset 0. In contrast, in second characteristic curve 

10 22 of the exposure sensitivity the offset has a value of +100, while in third characteristic 

curve 24 of the exposure sensitivity it has a value of -100. In both second characteristic curve 
22 of the exposure sensitivity and third characteristic curve 24 of the exposure sensitivity, the 
gain is 1. In fourth characteristic curve 26 of the exposure sensitivity, the gain is 2 with an 
offset of 0. A change in the offset shifts the characteristic curve of the exposure sensitivity in 

15 the ordinate direction, while a change in the gain and/or a change in the integration time 
expands or compresses the characteristic curve of the exposure sensitivity in the ordinate 
direction. A freely configurable characteristic curve of the exposure sensitivity in linear 
segments, such as for example first characteristic curve 20 of the exposure sensitivity in 
Figure 2, results in more degrees of freedom when adjusting at least one pixel of the image 

20 sensor, since here single segments can be adjusted individually. Along with the size and 
position of characteristic curve 20, 22, 24, 26 of the exposure sensitivity, this makes it 
alternatively or additionally possible to change the shape of characteristic curve 20, 22, 24, 
26 of the exposure sensitivity. 

Figure 3 shows a block diagram of the preferred exemplary embodiment, made up of an 
25 image sensor 40 and a processing unit 42 having various modules 50, 52, 54, 56. In the 
preferred exemplary embodiment, image sensor 40 is located behind the windshield of a 
motor vehicle in the area of the interior rearview mirror. In a variant of the preferred 
exemplary embodiment, image sensor 40 is built into the bumper and/or a side mirror and/or 
the taillight. Image sensor 40 is aligned in such a way that the image capture zone includes at 
30 least part of the surroundings of the motor vehicle. In the preferred exemplary embodiment, a 
black/white CMOS image sensor having a resolution of 640x480 pixels with 8-bit gray scale 
resolution is used as image sensor 40. Image sensor 40 has a freely configurable 
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characteristic curve of the exposure sensitivity in linear segments, the number of segments 
and/or the position of the segments and/or the size of the individual segments and/or at least 
one parameter of the linear functions in a segment being adjustable. Alternatively or in 
addition, the gain and/or the offset and/or the integration time are adjusted. In the preferred 
5 exemplary embodiment, the setting of the characteristic curve of the exposure sensitivity is 
adjusted jointly for all pixels of image sensor 40. Individual modules 50, 52, 54, 56 of 
processing unit 42 are implemented in the preferred exemplary embodiment in a digital 
microprocessor, which implements the functions of modules 50, 52, 54, 56 described below 
as programs, subprograms, or program steps. In one variant there are at least two 

10 microprocessors, to which individual modules 50, 52, 54, 56 are distributed. Of course it is 
also possible to distribute the individual modules to discrete processing units with their own 
computing capabilities. The image signals generated by image sensor 40 are transmitted over 
signal line 44 to processing unit 42. The image signals are transmitted electrically and/or 
optically and/or by radio. From the incoming image signals, i.e., the camera images, the 

15 histogram of the gray values is determined in histogram determination module 50. The 

histogram of the gray values indicates the frequency of the individual gray values of an image 
or image detail. In the preferred exemplary embodiment, the histogram of the gray values is 
determined from an overall image. For each gray value, the associated illumination intensity 
is determined in freely definable, virtual units in histogram determination module 50 as a 

20 function of the parameters set, such as the gain and/or the offset and/or the integration time, 
and/or the parameters for adjusting the characteristic curve of the exposure sensitivity, such 
as the number of segments of the characteristic curve of the exposure sensitivity, and/or the 
position of the segments of the characteristic curve of the exposure sensitivity, and/or the size 
of the segments of the characteristic curve of the exposure sensitivity, and/or the at least one 

25 parameter of the linear functions. In module 52 for determining the optimal characteristic 

curve, the optimal characteristic curve is calculated as a function of the histogram of the gray 
values determined in preceding histogram determination module 50. With N gray values in 
the histogram, having respective frequencies hj and associated illumination intensities <Dj, the 
result is a table of N elements, whose entries i are calculated for example from 
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where Gma X designates the maximum depictable gray value. The calculated function for 
mapping illumination intensities <D on gray values g represents the optimal characteristic 
curve of the exposure sensitivity according to information theory for the given histogram, and 
is an optimal characteristic curve of the exposure sensitivity. The optimal characteristic curve 
5 of the exposure sensitivity is distinguished basically by the fact that after adjustment of the 
characteristic curve of the exposure sensitivity of the image sensor, which is approximated to 
the determined optimal characteristic curve of the exposure sensitivity, the frequency of the 
individual gray values in the histogram of the image signals, i.e., of the camera image of the 
image sensor, is approximately constant, and/or the gray value density of the histogram of the 

10 image signals, i.e., of the camera image of the image sensor, is approximately constant. The 
frequency of a gray value here designates the number of pixels within the camera image that 
have this gray value based on the total number of pixels. A constant frequency of the gray 
values within a histogram of an image is referred to as a uniformly distributed histogram. The 
gray value density designates the sum of frequencies h(gi) of gray values gj in an interval Ag 

15 of gray values in reference to this interval Ag: (Z h(gi))/Ag. In module 54 for approximating 
the characteristic curve of the exposure sensitivity, the characteristic curve of the exposure 
sensitivity in linear segments of the pixels of the image sensor is now chosen so that it agrees 
as precisely as possible with the optimal characteristic curve obtained. In the preferred 
exemplary embodiment, this is performed in module 54 for approximating the characteristic 

20 curve of the exposure sensitivity, through a numerical minimization of the quadratic 
separation between the optimal characteristic curve of the exposure sensitivity and the 
characteristic curve in linear segments, while varying the parameters. Alternatively or in 
addition, in a variant of the preferred exemplary embodiment, the segmenting method 
described in Hwa-Cho Yi: Sensordatenauswertung mit Fuzzy-Logik fur das automatisierte 

25 Entgraten [Sensor data evaluation with fuzzy logic for automated deburring] (Munich: 

Hanser-Verlag, 1993, TU-Berlin dissertation, pp. 76-79) is used. This segmenting method 
operates according to the iterative final point method or the principle of the search for the 
longest segment. The determined parameters, such as the gain and/or the offset and/or the 
integration time and/or number of segments of the characteristic curve of the exposure 

30 sensitivity and/or the position of the segments of the characteristic curve of the exposure 
sensitivity and/or the size of the segments of the characteristic curve of the exposure 
sensitivity, and/or at least one parameter of the linear functions, are transmitted from module 
54 for approximating the characteristic curve of the exposure sensitivity to module 56 for 
generating the adjustment signals. In module 56 for generating the adjustment signals, the at 
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least one corresponding adjustment signal is determined from the determined parameters, and 
the at least one adjustment signal is transmitted over signal line 46 electrically and/or 
optically and/or by radio to image sensor 40. In image sensor 40, the characteristic curve of 
the exposure sensitivity is adjusted as a function of at least one adjustment signal. In the 
5 preferred exemplary embodiment, the characteristic curve of the exposure sensitivity takes 
effect in the image that follows determination of the optimal characteristic curve of the 
exposure sensitivity. Alternatively or in addition, in additional variants the characteristic 
curve of the exposure sensitivity takes effect in the same image and/or in the next image but 
one and/or in at least one additional image. 

10 The use of the method according to the present invention and/or of the processing unit 

according to the present invention and/or of the computer program according to the present 
invention for adjusting the characteristic curve of the exposure sensitivity of at least one pixel 
of at least one image sensor becomes clear when the at least one image sensor receives an 
incoming image having a wedge of gray values, the wedge of gray values being made up of 

15 two segments having a different gradient of the gray values. In otherwise adjusted and/or 
controlled and/or regulated image sensors having a characteristic curve in linear segments, 
with variable overall illumination of the gray value wedge, an apparent contour caused by the 
bent characteristic curve of the exposure sensitivity appears, which does not agree with the 
segment boundary of the gray value wedge. In contrast, through the method according to the 

20 present invention and/or the processing unit according to the present invention and/or the 
computer program according to the present invention, this apparent contour will lie exactly 
on or at least very close to the separation line, even if the overall illumination is changed. 

The described method and/or processing unit and/or computer program are not restricted to 
adjusting the characteristic curve of the exposure sensitivity of an image sensor in a motor 

25 vehicle. Rather, the described approach using the corresponding features is also usable 
outside of motor vehicle technology, moreover with more than one image sensor. 
Furthermore, the described method and/or processing unit and/or computer program is not 
restricted to adjusting the characteristic curve of the exposure sensitivity. Rather, the 
approach is also suitable in particular for controlling and/or regulating the characteristic curve 

30 of the exposure sensitivity. 

In a variant of the described method and/or the processing unit and/or the computer program, 
the characteristic curve of the exposure sensitivity of at least one pixel of at least one image 
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sensor is adjusted as a function of at least one image detail from at least one image sensor. A 
first embodiment includes the provision that the image segment is specifiable manually by at 
least one user, while in an additional embodiment the image detail is determined 
automatically by the processing unit. 

Another variant contains the provision, alternatively or in addition, that the characteristic 
curve of the exposure sensitivity of at least one pixel of at least one image sensor is adjusted 
as a function of image signals, in particular of at least one image and/or at least one image 
detail, from at least two or more image sensors. Alternatively or in addition, the adjustment of 
the characteristic curve of the exposure sensitivity of at least one pixel of at least one image 
sensor is performed as a function of image signals from at least one stereo camera. The stereo 
camera is made up here of at least two image sensors, which record essentially the same 
scene. For example, a common histogram of the gray values is calculated here from the 
images and/or image details and is used to determine the optimal characteristic cuiVe of the 
exposure sensitivity. 

15 In general, the described procedure is not restricted to a black/white CMOS image sensor 
having a resolution of 640x480 pixels with 8-bit gray scale resolution. Rather, the method 
and/or the processing unit and/or the computer program are suitable for all types of image 
sensors. The only prerequisite is that the image sensors have a configurable characteristic 
curve of the exposure sensitivity, having segments with functions, in particular continuous 

20 and/or linear functions. For example, in one variant a CCD image sensor is used. In another 
variant, alternatively or in addition, an image sensor having a logarithmic characteristic curve 
in segments is used. In addition, the described approach using the corresponding features is 
also usable with at least one color image sensor. Alternatively or in addition, in another 
variant the characteristic curve of the exposure sensitivity of at least one pixel of at least one 

25 image sensor is adjusted. 
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